The Role of Uncertainty in Aerospace Vehicle Analysis and Design 


Sean Kenny 

NASA Langley Research Center, Hampton, VA 
Luis Crespo 

National Institute of Aerospace, Hampton, VA 


Vision 

Increase confidence and consistency in performance robustness assessments by improving 
methods and software tools for quantifying and managing uncertainty. 

Effective uncertainty quantification (UQ) begins at the earliest phase in the design phase for 
which there are adequate models and continues tightly integrated to the analysis and design 
cycles as the refinement of the models and the fidelity of the tools increase. It is essential that 
uncertainty quantification strategies provide objective information to support the processes of 
identifying, analyzing and accommodating for the effects of uncertainty. Assessments of 
uncertainty should never render the results more difficult for engineers and decision makers to 
comprehend, but instead provide them with critical information to assist with resource utilization 
decisions and risk mitigation strategies. Success would be measured by the tools to enable 
engineers and decision makers to effectively balance critical project resources against system 
requirements while accounting for the impact of uncertainty. 

Historical Perspective 

The treatment of uncertainty quantification in physics-based models of aerospace systems has 
historically focused on their reliability analysis and is typically conducted in the final stages of 
the design cycle. Despite of its relatively late arrival to the analysis and design processes, this 
phase of application of UQ has proven to have significant impact on the final robustness and 
safety of the resulting vehicle. The conceptual design process is typically devoid of uncertainty 
quantification, and if done, it is usually only a crude assessment of global properties of the 
system. Of course, some of the primary challenges to performing UQ at the conceptual design 
stage are the lack of adequate representations of uncertainty, the lack of fidelity in the models 
representing the systems, and the need for an integrated multidisciplinary analysis environment. 
Although we feel that development of methods and tools to address conceptual level design and 
analysis may yield significant benefits, our background and experience is more aligned with 
application of UQ to the later design phases and will be the focus of the remainder of this 
document. 

As mentioned, most UQ work is conducted after the design and models have reached a minimum 
level of fidelity and maturity. Most government and industry settings employ random sampling 
together with simulation to assess a given design for a set of assumed uncertainties. The 
approach, commonly referred to as Monte Carlo analysis, has been successfully applied to many 
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systems and as such has the heritage necessary to serve, still today, as the baseline tool for the 
bulk of uncertainty quantification used within NASA and industry. Monte Carlo has at its core 
simplicity of implementation as it does not require rewriting source code and in fact can easily 
interface with most analysis codes. However, with this simplicity come fundamental limitations. 
One such limitation is the efficient identification of high-consequence low-probability events. 
This computational burden also makes it unsuitable for design, as most design procedures are 
based upon some type of search mechanism that requires repeated function evaluations, each 
requiring a separate Monte Carlo analysis. Furthermore, a significant limitation of Monte Carlo 
is that it requires the definition of probability density (joint or marginal) functions (PDF) for all 
uncertainties. A large amount of data is required to adequately prescribe a PDF-based uncertainty 
model. In many cases, sufficient data are not available and engineering judgment or expert 
opinion must be employed. This is particularly true for systems with limited or no heritage, cases 
in which UQ is performed utilizing models that have not been properly validated. Monte Carlo 
does not directly have the ability to access the impact of these modeling assumptions on the final 
UQ results. Yet another limitation of Monte Carlo is its inability to identify worst-case 
uncertainty combinations, to identify the dominant uncertain parameters and to evaluate the 
separation between the nominal operating conditions point and the failure event. 

Another critical need within the aerospace community is for design tools and methods to 
synthesize robust aerospace vehicles in the presence of uncertainties. There has been some effort 
in this area, but many approaches simply cannot handle the computational burden associated 
with propagating uncertainties through computationally expensive models in an iterative design 
loop. This certainly poses a difficult challenge, and in no way are we suggesting that we can 
radically redefine the fundamental limitations, but over the past several years, we have 
developed various methods for efficiently propagating uncertainties though nonlinear models and 
feel that we are well-positioned to make significant progress in this area. The following is a list 
of some of the techniques that we have developed and successfully applied to robust control 
design problems: moment propagation methods for efficiently estimating statistical performance 
of closed-loop systems, formulation of reliability-based performance metrics and their use in 
control design, homothetic deformations for failure domain bounding together with conventional 
design optimization for robustness enhancement, and more recently adaptive response surface 
technology to alleviate some of the computational expense. 

It is fully understood that models will always be uncertain and data too sparse to perfectly 
prescribe the physical phenomenon of interest, however a goal must be to explore and deploy 
alternate methods to conventional Monte Carlo analysis to better address its fundamental 
limitations. 


New Tools in UQ 

The primary analysis task in UQ is robustness and performance analysis. This task requires a set 
of analysis tools to efficiently assess robustness margins in the presence of parametric 
uncertainties. These uncertainties may be probabilistic in nature or may be defined by 
multidimensional bounded regions in the parameter space. Typical robustness and performance 
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assessments may include computing: probability of failure, mean system response, standard 
deviations, or bounded regions for guaranteed safe operation similar to a flight envelope. 

Recent work by the authors has resulted in a suite of efficient methods for uncertainty 
propagation and quantification of robustness for systems subject to parametric uncertainty. These 
unique tools can efficiently estimate the most likely behavior of the system, as well as the 
probability of rare, high-consequence events, as well as identify the most significant contributors 
to performance degradation (overall mission risk) caused by uncertainty. One such tool is failure 
set bounding as shown in Figure 1. The result in Figure 1 illustrates a system with two 
requirements (red and green lines), which separate the safe domain (white region) from the 
failure domain (gray region). This figure illustrates the maximal set, that for a given geometry 
and center, captures the maximal size object whose interior, by construction, resides in the safe 
domain. Techniques have been developed to use these maximal sets to compute upper bounds on 
failure probability, resulting in substantial computational saving over sampling-based methods. 
Maximal sets can also be used to determine the worst-case combinations of parameters, denoted 
byQin Figure 1. Unlike Monte Carlo, the computational burden of failure set bounding is 
independent of failure probabilities, thereby making it ideally suited for aerospace applications. 



Figure 1: Failure Set Bounding in Two Dimensions 


Design in the Presence of Uncertainty 
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Aerospace problems commonly involve complex, interdependent systems whose physics are 
multi-disciplinary in nature. Because of this complexity, the design of such systems, i.e., the 
process of prescribing the value of the system’s design variables so that requirements are met, is 
usually performed using deterministic parameter realizations. Examples of these design variables 
are control gains, geometric properties, and those that characterize a particular structural 
material. Examples of their requirements are closed-loop stability, maximum allowable weight, 
and maximum allowable cost. The challenge faced by the designer is to find a design point that 
satisfies conflicting design objectives while being robust to uncertainties. 

Since most current design practices do not directly accommodate for uncertainty it is quite 
possible that the performance degradation caused by it will render the design unacceptable. In 
such a case, the discipline expert faces the challenge of redesigning the system without having 
the knowledge, methods, and tools required to make the design sufficiently robust/reliable. This 
situation may lead to unnecessary design-analysis iterations and likely overly conservative 
solutions. 

Ideally, a framework would be developed that integrates the design and analysis processes 
thereby eliminating the costs associated with the prolonged generation of a satisfactory design as 
well as those of operating an overly conservative solution, e.g., excessive weight, poor 
performance under nominal operating conditions, higher manufacturing costs, etc. Robust design 
tools will enable the discipline expert to systematically pursue engineering solutions with the 
optimal robust/reliability characteristics. 

The integrated framework of the failure set bounding approach has been shown to facilitate 
robust design. This approach enables the discipline expert to pose the problems in a design- 
optimization setting from where engineering solutions with optimal robust/reliability 
characteristics can be systematically pursued. For a given uncertainty model, this would enable 
searching for the design points that (i) minimize the probability of violating the design 
requirements, (ii) minimize/maximize the expected value of a performance function, or (iii) 
minimize the variability in the system’s performance for a range of uncertainties. 

Figure 2 and Figure 3 present an example of robust design of a control system for the NASA 
Fangley subscale Generic Transport Model (GTM). The colors displayed in these figures 
indicate the number of requirements that are violated at any parameter realization of pitch 
stiffness and roll damping uncertainty. The color green indicates zero requirements violations, 
progressing from yellow (one violation) to black (nine violations). Figure 2 shows a robustness 
analysis of the nominal controller for the GTM. Notice that the points labeled A and B indicate 
stable and unstable system responses respectively. Figure 3 presents a redesigned controller that 
robustly accommodate for the uncertainty. Notice that the redesigned controller has a much 
larger safe operating domain and that point B is now within the safe domain resulting in a 
controller with greatly improved robustness characteristics. 
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Pitch stiffness uncertainty Pitch stiffness uncertainty 


Nominal Controller: Pilot tested & approved 



Roll damping uncertainty 


Figure 2: Before Robust Design 


Tuned Controller: Pilot tested & approved 
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Roll damping uncertainty 


Figure 3: After Robust Design 




The analysis and design of aerospace systems, where multiple uncertain parameters and various 
conflicting design requirements are present, are tasks of great practical importance that pose 
formidable challenges to the state of both the art and the practice. The aerospace industry as a 
whole will considerably benefit from the development of methods and tools that address these 
needs in a computationally efficient framework. 
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